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EXAMINER'S AMENDMENT 

An examiner's amendment to the record appears below. Should the changes and/or 
additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 
1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the 
payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview with 
Brian G. Hart (Reg. No. 44,421) on October 31, 2007 and November 1, 2007. 

The application has been amended as follows: 

Claims 1-40 were originally pending. Please amend claims 1 1, 16, 18-27, 33 and 37. 
Kindly cancel claims 15, 17, 32, 34-36 and 38 without prejudice. No claims are added or 
withdrawn. Accordingly, claims 1-14, 16, 18-31, 33, 37, and 39-40 remain pending. 
The following listing of claims replaces all prior versions and listings of claims in the 
application. 

1 . (Previously presented) A process implemented by a computing device for comparing two 
digital images, the process comprising: 

comparing texture moment data for the two images to provide a first similarity index; 

comparing color correlogram data for the two images to provide a second similarity 

index; 

comparing color moment data for the two images to provide a third similarity index; 
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combining the first, second and third similarity indices to provide a similarity value; 
determining that the two images match when the similarity value exceeds a first threshold 
value; and 

responsive to determining that the two images match, providing an indication that the two 
images match. 

2. (Original) The process of claim 1 , wherein combining comprises forming a weighted sum 
from the first, second and third similarity indices. 

3. (Original) The process of claim 1, further comprising determining* that the two images do 
not match when the similarity value is less than a second threshold value. 

4. (Original) The process of claim 1, wherein the two images comprise a first and a second 
image, the first of the two images comprises a query image, and further comprising locating one 
or more matches to a plurality of images in a database of images by iterating the acts of 
comparing color correlogram data, comparing color moment data, comparing texture moment 
data, combining and determining using as the second image a successive one image of the 
plurality of images. 

5. (Original) The process of claim 1, further comprising: 

comparing one or more high-level features from each of the two images; and 
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adjusting the similarity value in accordance with comparing one or more high-level 
features, prior to determining. 

6. (Original) The process of claim 1, further comprising: 

extracting one or more high-level features from each of the two images, wherein the one 
or more high-level features include a high-level feature chosen from a group consisting of: 
presence/absence of one or more human faces, indoor/outdoor scene and time/date of image 
formation; 

comparing one or more of the high-level features from each of the two images; and 
adjusting the similarity value in accordance with comparing one or more high-level 
features, prior to determining. 

7. (Original) The process of claim 1 , further comprising, after combining and before 
determining, adjusting the similarity value based on a calculation of commonality or lack of 
commonality between one or more high-level features associated with each of the two images 
and a confidence value thereof 

8. (Original) The process of claim 1, wherein at least one of the two images is a digital 
image taken using a digital camera. 



9. 



(Original) The process of claim 1, wherein combining comprises: 
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weighting the first, second and third similarity indices by assigning the first similarity 
index a greatest weight and assigning the third similarity index a least weight; and 

adding the weighted first, second and third similarity indices to provide an adjusted 
similarity value, wherein determining employs the adjusted similarity value in determining that 
the two images match. 

10. (Original) The process of claim 1, further comprising quantizing the similarity value prior 
to determining, wherein quantizing comprises quantizing according to a tiered ranking scheme 
employing fewer than ten levels ranging from a lowest to a highest tier, with greater similarity 
values corresponding to higher tiers, wherein the first threshold value corresponds to a highest 
tier into which similarity values are quantized. 

1 1 . (Currently amended) A process implemented by a computing device for manipulating 
image characterization data from a digital image, the process comprising: 

indexing the digital image by extracting low-level feature data corresponding to a 
plurality of low-level features from the digital image, the plurality comprising two or more low- 
level features chosen from a group consisting of: comprising texture moment data^ color 
correlogram data^ and color moment data; 

organizing the data into a feature vector by; . 

(a) forming a query vector; 

(b) if extracted low-level feature data corresponds to: 
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(i) texture moment data: comparing texture moment data for the query and 
feature vectors to provide a first similarity index; 

(ii) color correlogram data: comparing color correlogram data for the 
query and feature vectors to provide a second similarity index; and 

(iip color moment data: comparing color moment data for the query and 
feature vectors to provide a third similarity index; and 

(c) combining available ones of the first, second and third similarity indices to 
provide the feature vector; 

utilizing the feature vector to identify a set of similar digital images; and 
providing a result that identifies the set of similar images. 

12. (Original) The process of claim 1 1 , wherein indexing comprises: 
first extracting texture moment data from the digital image; 
second extracting color correlogram data from the digital image; and 
third extracting color moment data from the digital image. 

13. (Original) The process of claim 1 1 , further comprising storing the feature vector in a 
database. 

14. (Original) The process of claim 1 1 , wherein organizing the data comprises forming a 
query vector, and further comprising: 
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obtaining a feature vector from a group of feature vectors corresponding to a plurality of 
digital images; 

comparing the query vector to the feature vector to provide a similarity value; 

determining when the similarity value exceeds a first threshold; and 

iterating obtaining, comparing and determining for each feature vector of the group. 

15. (Canceled) 

1 6. (Currently amended) A proc e ss computer-readable storage medium having embedded 
there on computer-program instructions for comparing two digital images, the computer-program 
instructions, when executed by a processor, for performing operations comprising: 

comparing texture moment data for the two images to provide a similarity index , the 
similarity index being a first similarity index ; 

comparing color correlogram data for the two images to provide a second similarity 

index; 

comparing color moment data for the two images to provide a third similarity index; 

combining the first similarity index with other data to provide a similarity value , the other 
data comprising the second and the third similarity indices ; 

determining a degree to which the two images match when the similarity value exceeds a 
first threshold value; and 

responsive to determining the degree to which the two images match, providing a result 
that indicates whether the two images match. 
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17. (Canceled) 

1 8. (Currently amended) The proc e ss computer-readable storage medium of claim 1 6, 
wherein combining comprises forming a weighted sum from the similarity index and other data. 

1 9. (Currently amended) The proc es s computer-readable storage medium of claim 1 6, further 
comprising determining that the two images do not match when the similarity value is less than a 
second threshold value. 

20. (Currently amended) The proc e ss computer-readable storage medium of claim 1 6, 
wherein the two images comprise a first and a second image, the first of the two images 
comprises a query image, and further comprising locating one or more matches to a plurality of 
images in a database of images by iterating the acts of comparing, combining and determining 
using as the second image a successive one image taken from the plurality of images. 

2 1 . (Currently amended) The proc es s computer-readable storage medium of claim 1 6, further 
comprising: 

comparing one or more high-level features from each of the two images; and 
adjusting the similarity value in accordance with comparing one or more high-level 
features, prior to determining. 
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22. (Currently amended) The proc e ss computer-readable storage medium of claim 16, further 
comprising: 

extracting one or more high-level features from each of the two images, wherein the one 
or more high-level features include a high-level feature chosen from a group consisting of: 
presence/absence of one or more human faces, indoor/outdoor scene and time/date of image 
formation; 

comparing one or more high-level features from each of the two images; and 
adjusting the similarity value in accordance with comparing one or more high-level 
features, prior to determining. 

23. (Currently amended) The proc e ss computer-readable storage medium of claim 16, further 
comprising, after combining and before determining, adjusting the similarity value based on a 
calculation of commonality or lack of commonality between one or more high-level features 
associated with each of the two images and a confidence value thereof. 

24. (Currently amended) The proc e ss computer-readable storage medium of claim 1 6, 
wherein at least one of the two images is a digital image taken using a digital camera. 

25. (Currently amended) The proc e ss computer-readable storage medium of claim 1 6, 
wh e r e in the similarity ind e x is a first similarity ind e x, and furth e r comprising wherein combining 
the first similarity index with the other data further comprises : 
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comparing color corr e logram data for th e two imag e s to provid e a s e cond similarity 
ind e x; and 

comparing color moment data for th e two imag e s to provid e a third similarity index; 
wh e r e in combining compris e s: 

weighting the first, second and third similarity indices by assigning the first similarity 
index a greatest weight and assigning the third similarity index a least weight; and 

adding the weighted first, second and third similarity indices to provide an adjusted 
similarity value, wherein determining employs the adjusted similarity value in determining that 
the two images match when the similarity value exceeds the first threshold value. 

26. (Currently amended) The proc e ss computer-readable storage medium of claim 1 6, further 
comprising quantizing the similarity value according to a tiered ranking scheme employing fewer 
than ten levels ranging from a lowest to a highest tier, with greater similarity values 
corresponding to higher tiers, wherein the first threshold value corresponds to a highest tier into 
which similarity values are quantized. 

* 27. (Currently amended) A camera comprising: 

an image capture device configured to capture digital images; 

a memory coupled to the image capture device and configured to store a database of 
image data, including the digital images and associated feature vectors each comprising texture 
moment data, correlogram data and color moment data; and 

an image similarity module coupled to the memory and configured to: 
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generate a query vector comprising texture moment data , correlogram data and 
color moment data ; 

compare, in succession, the query vector to each feature vector in the database, to 
generate a similarity value; 

classify each feature vector to assign each feature vector to a tier in a tiered 
similarity structure based on the associated similarity value; 

group images associated with the respective feature vectors in each tier of the 
similarity structure to generate group(s) of similar images; and 

provide the group(s) of similar images for sorting. 

28. (Original) The camera of claim 27, wherein the image similarity module is further 
configured to: 

adjust each similarity value in conformance with comparison of high-level data 
corresponding to the query vector and high-level data corresponding to each feature vector to 
provide an adjusted similarity value corresponding to each feature vector; and 

use the adjusted similarity value to classify each feature vector in the tiered similarity 
structure. 

29. (Original) The camera of claim 27, wherein the image similarity module is further 
configured to adjust each similarity value in conformance with comparison of high-level data 
corresponding to the query vector and high-level data corresponding to each feature vector to 
provide an adjusted similarity value corresponding to each feature vector, wherein high-level 
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data comprises: presence/absence of one or more human faces; indoor/outdoor scene; and time of 
image formation. 

30. (Original) The camera of claim 27, wherein the image similarity module is further 
configured to: 

adjust each similarity value in conformance with comparison of high-level data 
corresponding to the query vector and high-level data corresponding to each feature vector to 
provide an adjusted similarity value corresponding to each feature vector, wherein high-level 
data comprises: presence/absence of one or more human faces, indoor/outdoor scene and 
time/date of image formation; and 

use the adjusted similarity value to classify each feature vector in the tiered similarity 
structure. 

3 1 . (Original) The camera of claim 27, further comprising a monitor configured to facilitate 
viewing of digital images stored in the database, wherein the digital images are accessed via 
associated feature vectors selected within a tier of the tiered similarity structure. 

32. (Canceled) 

33. (Currently amended) A computer readable storage medium on a tangible data storage 
d e vic e , the comput e r r e adabl e m e dium having embedded thereon a plurality of computer- 
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program instructions , the computer-program instructions, th e r e on that, when executed by one or 
more processors, caus e th e one or mor e proc e s s or s to for performing operations comprising : 

successively compar e comparing a query vector to each feature vector in a database, the 
query vector and the feature vectors each including texture moment data , correlogram data and 
color moment data ; 

successively compare texture moment data from the query vector to texture moment data 
from each feature vector to provide a first similarity index; 

successively compare correlogram data from the query vector to correlogram data from 
each feature vector to provide a second similarity index; 

successively compare color moment data from the query vector to color moment data 
from each feature vector to provide a third similarity index; and 

form a similarity value as a weighted sum of the first, second and third similarity indices 
for each feature vector, wherein the plurality of instructions that cause the one or more 
processors to rank comprise instructions that cause the one or more processors to quantize 
similarity values into ranked tiers based in part on high-level features associated with the query 
vector and the feature vectors, each tier corresponding to a different degree of similarity between 
the query vector and the feature vectors associated with that tier; 

*ank ranking each feature vector into a tier of a tiered classification scheme based on the 
comparison between the feature vector and the query vector; and 

utilizing the tiered classification scheme to provide an indication whether first and second 
digital images have matching similarities. 
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35. (Canceled) 

36. (Canceled) 

37. (Currently amended) A system for image correlation comprising: 
one or more processors; 

a memory coupled to the one or more processors , the memory comprising computer- 
program instructions executable by the one or more processors and configur e d to store a database 
representing digital images , the computer-program instructions comprising ; 

an image comparison module configured to 

(a) cause the one or more processors to access the data representing the images 
and to select one or more feature vectors each associated with a corresponding image based on 
comparison of color texture data and other data contained in each feature vector with analogous 
data contained in a query feature vector; and 

(b) compare correlogram data and color moment data contained in each feature 
vector with analogous data contained in the query feature vector; and 

(c) select the one or more feature vectors based on a weighted sum of similarity 
indices respectively derived from comparing the color texture moments, the correlogram data 
and the color moment data; and 
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a display device coupled to the one or more processors and to the memory and configured 
to display digital images corresponding to the selected one or more feature vectors in response to 
user input, the digital images having been determined to share a set of similarity characteristics. 

38. (Canceled) 

39. (Original) The system of claim 37, wherein the other data include high-level feature data. 

40. (Original) The system of claim 37, wherein the image comparison module further is 
configured to cause the one or more processors to: 

extract one or more high-level features contained in each feature vector and analogous 
data contained in the query feature vector, wherein the one or more high-level features include a 
high-level feature chosen from a group consisting of: presence/absence of one or more human 
faces; indoor/outdoor scene; and time of image formation; 

compare one or more of the high-level features from each feature vector and analogous 
data contained in the query feature vector; and 

employ results from comparing the one or more high-level features in selecting the one or 
more feature vectors. 

41. (Canceled) 



1 
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DETAILED ACTION 

1 . This action is responsive to the telephonic interview on 1 0/3 1 /07 and papers filed on 
7/26/07. 

Response to Amendment 

2. In response to applicant's amendment received on 7/26/07, all requested changes to the 
claims have been entered. 

Allowable Subject Matter 

3. Claims 1-14, 16, 18-31, 33, 37 and 39-40 are allowed. 

The following is an examiner's statement of reasons for allowance: 

4. As to claims 1, 1 1, 16, 27, 33 and 37, none of the prior art teach or fairly suggests using 
two or more of the comparing texture moment data of two images to generate a first similarity 
index, comparing color correlogram data of two images to generate a second similarity index, 
comparing color moment data of two images to generate a third similarity index, and combining 
the indices to provide a similarity value. The prior art of USPN 6,957,387 to Barbieri discloses 
comparing on color, texture, shape or motion of two images to determine a similarity and to 
determine if the two images match (column 6, lines 15-51), including the use of color 
correlogram data (column 11, lines 45-51), but does not teach or fairly suggest comparing the 
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images based on two or more of the image feature data and creating an combined similarity 
value for use in determining if they match. The prior art of US 2003/0093437 to Gargi et al. 
discloses determining a distance between two images based on several weighted features 
determined from each of the two images, the features including HSV histogram, color moment, 
color coherence, MRSAR texture, coarseness and directionality data (Tables 1-4). However, 
Gargi et al. does not teach or fairly suggest using the distance to determine a similarity value 
from which it is determined that the two images match. The prior art of the article titled 
"Benchmarking of Image Features for Content-based Retrieval" to Ma et al., already or record, 
discloses the advantages of using each of the features, including color moment data, color 
correlogram data and texture moment data, individually for image retrieval (sections 3.3 and 
3.4), but does not teach or fairly suggest comparing the images based on two or more of the 
image feature data and creating an combined similarity value to determine if they match. In 
conclusion, none of the prior art teaches or fairly suggest using two or more of the comparing 
texture moment data of two images to generate a first similarity index, comparing color 
correlogram data of two images to generate a second similarity index, comparing color moment 
data of two images to generate a third similarity index, and combining the two or more generated 
indices to provide a similarity value, as disclosed in the limitations of claims 1, 11, 16, 27, 33 
and 37. 

Any comments considered necessary by applicant must be submitted no later than the 
payment of the issue fee and, to avoid processing delays, should preferably accompany the issue 
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fee. Such submissions should be clearly labeled "Comments on Statement of Reasons for 
Allowance." 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

USPN 6,246,790 to Huang et al. discloses image indexing using color correlograms. 
USPN 7,233,708 to Li et al. discloses image indexing and retrieval using texture moment 

data. 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Aaron W. Carter whose telephone number is (571) 272-7445. 
The examiner can normally be reached on 8am - 4:30 am (Mon. - Fri.). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (571) 272-7453. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




